• PHASE 1: Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 70%)

• PHASE 2: ECOG performance status =\< 2 (Karnofsky \>= 60%)

• Unless a patient has had orchiectomy by surgery, the patient is expected to be on antiandrogen therapy (ADT) for medical castration. ADT needs to be maintained throughout the study. Testosterone level should be checked, and kept consistently lower than 50 ng/dL, similar to that obtained with bilateral orchiectomy

• Progressive castration-resistant prostate cancer with two or more skeletal metastases identified by 99mTC bone scintigraphy. One or more lymph node metastases allowed, but not mandatory. Lymph node metastases in each individually must measure less than 3 cm in the longest dimension. Visible visceral organ metastases are not allowed. A diagnosis of prostate cancer must have been histologically confirmed at any time point

• Baseline prostatic specific antigen (PSA) level of 1 ng/mL or higher with evidence of progressively increasing PSA values (two consecutive increases over the previous reference value)

• Progression after at least one of the following: abiraterone, enzalutamide, apalutamide, darolutamide, or taxane chemotherapy (docetaxel, cabazitaxel). There is no maximum number of prior therapies. Prior immunotherapies (for example, Sipuleucel-T or pembrolizumab) do not exclude the patient from participation

• Age \>= 18 years. Castrate-resistant prostate cancer (CRPC) affects older adults and is rarely encountered in children and adolescents

• Life expectancy \>= 6 months

• Albumin \> 2.5 mg/dL

• Hemoglobin \> 9 g/dL

• Leukocytes \>= 3,000/mcL

• Absolute neutrophil count \>= 1,500/mcL

• Platelets \>= 100,000/mcL

• Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (with the exception of \< 3 mg/dL for patients with Gilbert's disease)

• Aspartate aminotransferase (AST)(serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT)(serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x institutional ULN

• Creatinine =\< 1.5 x institutional ULN OR

• Glomerular filtration rate (GFR) \>= 40 mL/min/1.73 m\^2

• Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy (with no medications prohibited by this protocol \[e.g. drug-drug interactions\]) with undetectable viral load within 6 months are eligible for this trial

• For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy (with no medications prohibited by this protocol \[e.g. drug-drug interactions\]), if indicated

• Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load (with no medications prohibited by this protocol \[e.g. drug-drug interactions\])

• Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial

• Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional classification. To be eligible for this trial, patients should be class 2B or better

• Concomitant use of physiologic corticosteroids is allowed

• Concomitant use of bisphosphonates is allowed (use of bone health agents is mandatory - either denosumab \[preferred\] or bisphosphonates)

• The effects of radium-223 dichloride, M3814, and avelumab on the developing human fetus are unknown. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 6 months after completion of Radium-223 dichloride, M3814, and avelumab administration

• Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity (IDMC) who have a legally-authorized representative (LAR) and/or family member available will also be eligible

• Patients must be able to swallow orally administered medication

• Patients with asymptomatic, treated brain metastases are permitted if there is no evidence of progression for at least 4 weeks after central nervous system (CNS)-directed treatment, as ascertained by clinical examination and brain imaging (magnetic resonance imaging \[MRI\] or CT scan) during the screening period